Virtual Reality Training Application for Surgical Scrubbing-in Procedure

ABSTRACT

Disclosed herein are immersive virtual and/or augmented reality education/training tools useable to teach operating room personnel any or all of scrubbing-in procedures, gowning/gloving procedures, and proper operating room etiquette. The training tools provide individuals with active practice in the operating room setting prior to the real-world applications, and thus allows the user to become more adept with the necessary procedures prior to entering the operating room. The disclosed virtual and/or augmented reality training will make for a better, less stressful, risk-reducing (e.e., infection of patient), operating room environment that is more conducive for both learning by the students and surgical performance by the experienced operating room personnel.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit under 35 U.S.C. § 119(e) ofprior U.S. Provisional Application No. 62/591,718 filed Nov. 28, 2017,the disclosure being incorporated herein by reference in its entirety.

TECHNICAL FIELD

This invention pertains generally to a software application providinginteractive training tools, and more specifically to a virtual/augmentedreality assisted learning aid that provides enhanced training for thesurgical scrubbing-in and gowning/gloving procedures and properoperating room etiquette.

BACKGROUND

Many healthcare professionals and healthcare professionals-in-traininghold positions that entail performing in an operating room (OR). Thisincludes but is not limited to physicians/surgeons, residents, medicalstudents, nurses, nursing students, operating room technicians, andoperating room technician students. The operating room must be anenvironment that allows for the fluid, efficient, and sterile completionof a surgical procedure. In order to ensure this, every person in theoperating room must adhere to very specific procedures and operatingroom etiquette before, during, and after every surgery. For many,especially novices to the OR, these procedures are very difficult tolearn in a real-world environment due to the fast pace and necessaryexactness of the techniques. This issue (gap in knowledge) is especiallytrue for the scrubbing, gowning and gloving procedures prior to surgery,also called the scrubbing-in procedure.

The purpose of scrubbing-in is to eliminate the controllable sources ofcontamination via the performance of aseptic procedures. For ORnewcomers, ignorance of proper scrubbing-in procedures is common. Notonly does this ignorance threaten the integrity of the operating room,but it also affects the natural flow of the operating room protocolcausing stress upon the trainees and the experienced OR personnel.

Prior methods for training have included detailed written protocols,and/or videos of the procedure. Neither, however, provides actualhands-on experience that may simulate the physical and psychologicalenvironment of the OR. The concept of simulation of critical events tohone skills, in contrast to mere book learning, has long been a stapleof human training methodology. At its heart, the goal of simulation isto truly mimic the physical and psychological experience of an event,thus harnessing the power of emotional context and psychological stressto retain both physical and intellectual skills and lessons with morereliability than reading about or viewing the procedure alone can yield.

Various industries have adopted and refined simulation-based trainingmethodologies, attempting to replicate work environments as preciselyand accurately as possible to prepare students and professionals forcritical events they may encounter in practice. In the aviationindustry, for example, flight simulators have improved over time ascomputer technology has become more advanced and affordable. In theinstitution of medicine, medical scenario simulation has grown to becomea standard component of medical training and continuing education,typically relying on physical “dummy” apparatuses to represent the“patients” or “subjects” of the simulation. Equipping and maintaining astate-of-the-art simulation facility employing such manikins representsa significant cost.

Simulation-based training systems that are both low cost and completelyimmersive are significantly limited or non-existent in many industries,and particularly in the medical industry. Accordingly, there exists aneed in the art for improved training methods for medical procedures andenvironments such as, for example, surgical scrubbing-in procedures andthe proper etiquette required in the OR environment.

SUMMARY

The presently disclosed invention provides an augmented and/or virtualreality systems and training methods for surgical scrubbing-inprocedures, gowning/gloving procedures, and the proper etiquette whilein the OR.

Thus, according to its major aspects, and briefly stated, the presentlydisclosed invention includes a computer-implemented method for providingan augmented and/or virtual reality training tool. The training toolincludes generation of an augmented or virtual reality environment;detection of actions within the augmented or virtual reality environmentcomparison of those actions to a performance metric; and generation,based at least in part on the comparison, of a performance evaluation.

According to certain aspects of the method, the augmented or virtualreality environment may be an OR environment, and the actions may beassociated with a surgical scrubbing-in procedure, a gowning/glovingprocedure, and/or aseptic performance during a surgical procedure. Assuch, the performance metric may detect actions taken during thescrubbing-in and/or gowning/gloving procedures, such as a proper orderof steps within each procedure, and the performance evaluation maydefine an aseptic quality of the procedures, a speed of the procedures,aseptic technique during the scrubbing-in and/or gowning/glovingprocedures or a surgical procedure, or any combination thereof. Theactions may be further associated with proper etiquette in the OR, andthe performance metric may be detect actions taken before, during, andafter a surgical procedure. According to certain aspects of the method,the actions may comprise a physical movement of a user of the system, amovement of a virtual or real object in the augmented or virtual realityenvironment, or any combination thereof.

The presently disclosed invention further includes acomputer-implemented program product for providing the augmented orvirtual reality training tool, the computer-implemented program productcomprising a non-tangible computer readable medium tangibly embodyingnon-transitory computer-executable program instructions thereon that,when executed, cause a computing device to execute the method forproviding an augmented or virtual reality training tool.

The presently disclosed invention further includes a system forproviding the augmented or virtual reality training tool. The system mayinclude a non-transitory memory comprising processor-executableinstructions; and a processor coupled to the non-transitory memory andconfigured to execute the processor-executable instructions, wherein theprocessor-executable instructions comprise instructions to implement theaugmented and/or virtual reality training tool. According to certainaspects, the system may further include a display in communication withthe processor, the display configured to provide visual representationof the augmented or virtual reality environment and visual feedback ofthe performance evaluation.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the present invention will be had uponreference to the following detailed description when read in conjunctionwith the accompanying drawings, wherein like numerals represent likefeatures in the various views. It is to be noted that features andcomponents in these drawings, illustrating views of embodiments of thepresent invention, unless stated to be otherwise, are not necessarilydrawn to scale.

FIG. 1 is a flow diagram of steps in the Delphi process used to provideelements of the scrubbing-in and gowning/gloving procedures according tovarious embodiments of the presently disclosed invention;

FIG. 2 shows a bar diagram of the mean of the Likert score importancevalue for each of the 43 steps surveyed in the second round of thesurvey process according to the Delphi method used in the presentinvention;

FIG. 3 shows a bar diagram of the mean consensus agreement for each ofthe 30 steps surveyed in the third round of the survey process accordingto the Delphi method used in the present invention; and

FIG. 4 is a block diagram of a training system according to variousembodiments of the presently disclosed invention.

DETAILED DESCRIPTION

In the following description, the present invention is set forth in thecontext of various alternative embodiments and implementations involvingaugmented and virtual reality methods and systems which provide novellearning and training tools. While the following description disclosesnumerous exemplary embodiments, the scope of the present patentapplication is not limited to the disclosed embodiments, but alsoencompasses combinations of the disclosed embodiments, as well asmodifications to the disclosed embodiments.

Various aspects of the systems and methods may be illustrated bydescribing components that are coupled, attached, and/or joinedtogether. As used herein, the terms “coupled”, “attached”, and/or“joined” are interchangeably used to indicate either a direct connectionbetween two hardware components, or two software modules, or, whereappropriate, an indirect connection to one another through interveningor intermediate components or modules. In contrast, when a component isreferred to as being “directly coupled”, “directly attached”, and/or“directly joined” to another component or module, there are nointervening elements shown in said examples.

Various aspects of the systems and methods may be described andillustrated with reference to one or more exemplary implementations. Asused herein, the term “exemplary” means “serving as an example,instance, or illustration,” and should not necessarily be construed aspreferred or advantageous over other variations of the devices, systems,or methods disclosed herein. “Optional” or “optionally” means that thesubsequently described event or circumstance may or may not occur, andthat the description includes instances where the event occurs andinstances where it does not. In addition, the word “comprising” as usedherein means “including, but not limited to”.

It must also be noted that as used herein and in the appended claims,the singular forms “a”, “an”, and “the” include the plural referenceunless the context clearly dictates otherwise. Unless defined otherwise,all technical and scientific terms used herein have the same meanings ascommonly understood by one of ordinary skill in the art.

Provided herein are systems, methods, and non-transitorycomputer-readable media for simulation based training. This methodologyuses augmented reality and/or virtual reality to greatly advance thedegree of environmental and psychological fidelity available in thesimulations. An augmented reality (AR) environment refers to theperception of a user of their real, physical environment with theaddition of virtual, projected, two or three dimensional objects in thatenvironment. Integral to the concept of an augmented reality environmentis the feature of the virtual objects to be perceived to exist in thereal space as if they were real objects, with the ability of users towalk around them and see them from different angles, as appropriate. Avirtual reality (VR) environment refers to a wholly virtual, projectedenvironment which generally includes images, sounds and other sensationsto replicate a real environment or an imaginary setting, and simulates auser's physical presence in this environment to enable the user tointeract with this space. In harnessing this for training, the methoddescribed enables a replication of nearly an infinite number ofenvironments.

Establishing an augmented reality environment within a real space refersto using computer generated virtual objects projected into the space,where the virtual objects behave as if they are physically in the space,and where one or more users may be able to see each other (i.e., actualor virtual representations) and the virtual objects, and interact withthe virtual objects and each other. Alternatively, virtual realityenvironments may be established independent of the space, and include anentirely virtual space and virtual objects. Virtual representation ofone or more users may interact with each other and with the virtualspace and objects therein.

A real-world operating room (OR) experience may be simulated using animmersive virtual or augmented reality. Each of the vital steps ofscrubbing, gowning, and gloving may be presented step-wise allowing theuser to choose the correct procedure amongst many other incorrectoptions. Moreover, OR etiquette may be modeled, providing an immersivetraining experience regarding proper interactions while in the OR. Forexample, standard interaction steps while in the OR may include, but arenot limited to, introduction to the scrub nurse and circulating nurse,assisting with the gowning and gloving procedure (e.g., working with thenursing staff to properly dry hands and determine gown and glove size,etc.), offering to help clean up and transfer patient while stillremaining sterile, being aware of what is sterile (e.g., everything thatis draped with blue or resting on a blue surface), etc.

The software application may provide feedback (i.e., immediate feedbackand/or delayed feedback) based on the user's selections and actions. Thefeedback may come in the form of virtual representations of OR personneland/or a patient during the simulation and/or at the end of thesimulation. Additionally, or alternatively, the feedback may come in theform of visual, audible, or haptic signals during the simulation and/orat the end of the simulation.

Thus, disclosed herein is an advanced medical procedure simulator whichmay be based on a virtual or augmented reality simulation apparatus. Thesimulator provides a means for medical professionals to experience thescrubbing-in procedure, the gowning/gloving procedure, and/or proper ORetiquette. The virtual or augmented reality system may compute anddisplay a visual virtual or augmented reality model of the ORenvironment, and in accordance with the user's (e.g., medicalprofessional's) gestures and actions, may provide feedback, such asvisual, audible, or haptic signals.

In a virtual reality system, an entirely virtual image may be simulatedfor display to a user, and in an augmented reality system, a simulatedimage may be overlaid or otherwise incorporated with an actual image fordisplay to the user. Various OR environments can be selected. Therefore,various different scenarios, such as would be encountered over the yearsby OR personnel, can be simulated for a user in a compressed period oftime for training purposes. The virtual or augmented reality simulationsystem can also compute and provide various metrics and statistics ofperformance.

As used herein, the terms “virtual” and “augmented” may be usedinterchangeably unless specifically indicated otherwise. As such,reference to an augmented reality system may include reference to eitheran augmented or virtual reality system. Moreover, reference to anaugmented reality display device may include reference to an augmentedor virtual reality display device.

During the simulation, the user may encounter OR equipment and learn ORetiquette. The course of the simulation session may be dependent uponthe actions of the user, such as button selections, voice inputs,movements of the user and/or virtual or real objects within thesimulation environment, etc. Moreover, the software application mayoffer a consequence display representative of the user's actions (e.g.what happens to a patient or future patients if the proper proceduresare not followed, such as the patient developing an infection).

Possible actions which the system of the present invention may belooking to record and compare include at least proper surgicalscrubbing-in procedure, gowning/gloving procedure, and/or aseptictechnique while in the OR during a surgical procedure. Such actions, orsequence of actions (steps) are a focus of the present invention, andare defined herein. These steps have been concisely defined throughstudies using the Delphi method, a method widely used and acceptedinvolving a structured communication technique for achieving convergenceof opinion from a panel of experts (Thangaratinam S, Redman C W. TheDelphi technique. The Obstetrician & Gynaecologist. 2005; 7:120-125).

The Delphi technique facilitates an efficient group dynamic process andis done in the form of an anonymous, written, multistage survey process,where feedback of group opinion is provided after each round. Ingeneral, the Delphi technique is performed in several stages thatinclude a first stage of convening a panel of experts about a topic ofinterest, such as surgeons convened to evaluate a scrubbing-inprocedure. In a second stage, input from these experts, typicallyprovided anonymously, is received in a structured way (e.g., answers toa questionnaire, an opinion on a defined problem, a set of ratingscales, etc.). Evaluation of the input may be completed using a set ofcriteria, wherein the input may be filtered and summarized if necessary.The summarized input may then be presented to the experts in asubsequent round of evaluation (e.g., additional questionnaire), givingthem an opportunity to comment and change their input based on theevaluation. This subsequent round of input may be evaluated andre-presented to the experts in a further survey round. These later stepsmay be iteratively repeated until the opinions of the experts arestable. It is expected that using the Delphi technique the range of theanswers will decrease and converge towards consensus among the experts.

In the present invention, consensus was defined as >85% agreement amongthe expert panelists. For example, to determine the steps in ascrubbing-in and gowning/gloving procedure as disclosed herein, a firststage of the Delphi process was executed by convening a panel of expertsfrom two hospitals across six different subspecialties. The panelmembers were chosen based on their experience as physicians in the OR,and each participated in various rounds of written survey (see Table 1).

TABLE 1 Specialty Round 1 Round 2 Round 3 Orthopedic 1 1 0 Urology 1 1 2Ob/Gyn 6 17 19 General 1 4 3 Plastic 0 5 4 Neurosurgery 0 1 0 Total 9 2928

With reference to FIG. 1, in a second stage of the Delphi process, twoopen-ended questions were administered as a written survey (step 10;first round). The survey questions were developed and reviewed by apanel of facilitators different from the survey panel of surgeons. Thepanelists' answers to the open ended questions served to provide a listof steps that were be used in further surveys to formulate a final listof steps for the scrubbing-in and gowning/gloving procedures. Each ofthe panelists responded to these questions anonymously (step 20). Theirresponses to this first questionnaire were then used to extract a listof more specific questions, i.e., list of steps, which were included ina second questionnaire (step 30). The panelists were then provided withthis second questionnaire (step 40; second round) and asked to rank thesteps on a Likert scale (1-5). A statistical analysis was performed,wherein a mean cutoff of >4.0 was used to refine the data (step 50). Therefined data, i.e. shortened list of steps, was then presented to thepanelists as a third questionnaire (step 60, third round), along withfeedback from analysis of the second round. The panelists were asked tomake a final decision to keep or remove each step from a final list ofsteps. This final list of steps that define the scrubbing-in procedureand/or the gowning/gloving procedure was modified and consolidated basedon an agreement percentage amongst panelists of at least 85% (step 70).The final outcome of the Delphi technique is a multi-step procedure forthe scrubbing-in and/or the gowning/gloving process according to thepresently disclosed invention.

In an exemplary Delphi process, as detailed herein, the first roundincluded two open-ended questions that lead to 43 total possible stepsthat may define the scrubbing-in and/or gowning/gloving procedure. These43 steps were presented to the expert panelists in a second round asdetailed above, wherein the steps were ranked on a Likert score of 1-5as shown in FIG. 2. The 30 steps having a Likert score of greater than4.0 were then presented to the expert panelists in a third round asdetailed above. These 30 steps were included/excluded from a finalscrubbing-in and/or the gowning/gloving procedure. A mean consensusagreement of at least 85% for each of the 30 steps (see FIG. 3) was usedto define the final 22 steps included in the multi-step procedure forthe scrubbing-in and gowning/gloving process according to the presentlydisclosed invention.

For example, a surgical scrubbing-in procedure according to the presentinvention may comprise the following 10 steps performed outside of theOR, such as in a “sink room”:

-   1. Remove jewelry from hands/arms.-   2. Put on face mask and eye protection.-   3. Grab and open a pre-packaged scrub/nail kit.-   4. Moisten hands and arms under the water without touching the    faucet.-   5. Use firm (bristled) side of brush to scrub nails.-   6. Use firm (bristled) end of scrub brush to scrub all surfaces of    fingers and hands.-   7. Use sponge to scrub the entire length of one forearm, from wrist    to elbow.-   8. Use sponge to scrub entire length of contralateral forearm, from    wrist to elbow.-   9. Rinse off both arms allowing water to fall from wrists to elbows.-   10. Use back/butt/hip to enter OR keeping arms up and away from    torso.

Once inside the OR, the following 12 steps may define a proper gowningand gloving procedure according to the present invention:

-   1. Enter OR with elevated hands/arms and approach scrub tech/nurse,    taking care to avoid touching anything.-   2. Hold out one hand to accept a dry towel from scrub tech/nurse.-   3. Dry opposite hand/arm using the hand in which the towel was    placed, from hand to elbow.-   4. Flip towel to dry hand and dry opposite hand/arm that has not yet    been dried, from hand to elbow.-   5. With scrub tech/nurse holding gown open, place both hands/arms    into sleeves allowing fingertips to come through ends of sleeves.-   6. Allow nonsterile nurse/circulator to tie up back of gown.-   7. With scrub tech/nurse holding right glove open, place hand into    right glove.-   8. With scrub tech/nurse holding left glove open, place left hand    into glove.-   9. Hand card (at end of gown tie) to scrub tech/nurse or circulator.-   10. Rotate in gown with scrub tech/nurse or circulator still holding    card.-   11. Re-grasp the tie from the scrub tech/nurse or circulator.-   12. Tie both ties of gown together.

Moreover, once inside the OR, proper etiquette can also be modeled usingthe methods and systems of the present invention. For example, properaseptic technique during a surgical procedure may include keeping handsin the sterile field (i.e., sterile only touches sterile), whereinsterile areas of the gown are from below the neck to surgical tablelevel. Other actions may include: never turning your back to the sterilefield, remaining aware of everything that is sterile (e.g., blue drapesindicate sterile equipment/area), and always choosing to notify ORpersonnel if the sterile area is compromised—even if nobody is watching.

Additional actions which the system of the present invention may belooking to record and compare include at least selection of the propersurgical attire (e.g., both size and total list which may include, forexample, surgical cap, shoe covers, and mask). The actions may furtherinclude, for example, performing certain tasks in the OR, such asintroduction to the scrub nurse and circulating nurse, offering toobtain/open gown and gloves, proper technique to open gloves and gownfrom package, working together to determine glove size, setting asidephone, and writing name on whiteboard in the OR room so that ORpersonnel can identify you. Additional tasks may include, for example,remaining attentive to the “pre-induction verification” and “timeout”signals.

A “pre-induction verification” occurs well prior to the initiation ofthe surgery and is verbal acknowledgment to all personnel in theoperating room about any of the following: patient id, allergies,planned procedure (state the procedure to be performed), consentcorrect, patient position, site marked (the side of the body and sitethat the procedure is to be performed), antibiotics available, implants,devices, special equipment, etc.

A “timeout” occurs immediately before the start of the procedure,subsequent procedures, and/or a new attending enters, and includes butis not limited to: patient id, correct procedure, correct site/side,correct position, antibiotic started, preparation of the site is dried,etc.

Thus, according to certain aspects, the present invention relates to atraining method for a scrubbing-in procedure, wherein the method maycomprise identifying, using a processor, a plurality of steps of thescrubbing-in procedure, and presenting, using an augmented realitydisplay of an augmented reality device, a virtual representation of anOR environment. The OR environment may include a sink room of anoperating room suite, and an operating room of the operating room suite.The method may include querying a user for a first step in thescrubbing-in procedure, and determining whether the first step in thescrubbing-in procedure has been completed. In response to determiningthat the first step has been completed, the method may further includepresenting to the user a virtual representation of a subsequent step inthe scrubbing-in procedure.

The method may further include, before determining the first step in thescrubbing-in procedure has been completed, determining that the firststep is not being performed, or that the first step is being performedimproperly. The method may then display an alert using the augmentedreality display indicating that an error has occurred. The alert may bevisual, audible, or haptic. According to certain aspects, the alert maybe a text alert that shows within the augmented reality display, or mayinclude other forms of alert, such audible alerts (e.g., spoken languageor sound alerts).

The plurality of steps in the scrubbing-in procedure are ordered, andgenerally include: (1) removing jewelry from hands and arms; (2) puttingon a face mask and eye protection; (3) taking and opening a pre-packagedscrub/nail kit; (4) moistening hands and arms with water from a faucetwithout touching the faucet; (5) using a bristled side of a brush fromthe pre-packaged scrub/nail kit to scrub nails; (6) using a bristled endof the brush to scrub all surfaces of fingers and hands; (7) using asponge to scrub an entire length of one forearm, from wrist to elbow;(8) using the sponge to scrub an entire length of contralateral forearm,from wrist to elbow; (9) rinsing off both forearms allowing water tofall from wrists to elbows; and (10) using back, butt, or hip to enterthe operating room while keeping arms positioned up and away from torso.

The present invention also relates to a training method for agowning/gloving procedure, wherein the method may comprise identifying,using a processor, a plurality of steps of the gowning/glovingprocedure, and presenting, using an augmented reality display of anaugmented reality device, a virtual representation of an OR environment,generally including an operating room. The method may include querying auser for a first step in the gowning/gloving procedure, and determiningwhether the first step in the gowning/gloving procedure has beencompleted. In response to determining that the first step has beencompleted, the method may further include presenting to the user avirtual representation of a subsequent step in the gowning/glovingprocedure.

The method may further include, before determining the first step in thegowning/gloving procedure has been completed, determining that the firststep is not being performed, or that the first step is being performedimproperly. The method may then display an alert using the augmentedreality display indicating that an error has occurred.

The plurality of steps in the scrubbing-in procedure are ordered, andgenerally include: (1) entering the operating room environment withcleaned wet hands and arms in an elevated position, and approaching ascrub technician or nurse, taking care to avoid touching anything; (2)holding out first hand to accept a dry towel from the scrub technicianor nurse; (3) drying second hand and arm using the first hand, from handto elbow; (4) passing the towel to the second hand to dry the first handand arm, from hand to elbow; (5) placing both hands and arms intosleeves of a gown, allowing fingertips to come through ends of sleeves,wherein the gown is held open by the scrub technician or nurse; (6)allowing a nonsterile nurse or circulator to tie a back of the gownusing a first set of ties; (7) placing right hand into right glove,wherein the right glove is held open by the scrub technician or nurse;(8) placing left hand into left glove, wherein the left glove is heldopen by the scrub technician or nurse; (9) handing card (at end of asecond tie) to the scrub technician or nurse, or to the nonsterile nurseor circulator; (10) rotating in gown with the scrub technician or nurse,or to the nonsterile nurse or circulator still holding the card (at endof the second tie); (11) re-grasping the second tie from the scrubtechnician or nurse, or to the nonsterile nurse or circulator; and (12)tying the second tie at front of the gown.

According to certain aspects of the invention, the method may includeboth the scrubbing-in and gowning/gloving procedures. As such, once eachof the plurality of steps in the scrubbing-in procedure are completed,the method may include querying a user for a first step in agowning/gloving procedure.

The present invention also includes computer implemented processes forestablishing an augmented or virtual reality environment comprising anOR environment, and detecting actions of the user within the virtual oraugmented reality environment, such as actions on virtual objects (i.e.,generated in an AR/VR environment), or actions on real objects within areal OR environment (i.e., overlaid with a virtual OR environment suchas would be generated by AR technology). These actions may be recordedand evaluated against a performance metric to define a performanceevaluation for a specific procedure.

For example, in the OR environment, the procedure may comprise ascrubbing-in procedure, and the performance metric may provide aperformance evaluation of an aseptic quality of the scrubbing-inprocedure, a speed of the scrubbing-in procedure, performance of allsteps in the scrubbing-in procedure, performance of a proper order ofthe steps in the scrubbing-in procedure, or a combination thereof.

Moreover, the procedure may comprise a gowning/gloving procedure, andthe performance metric may provide a performance evaluation of anaseptic quality of the gowning/gloving procedure, a speed of thegowning/gloving procedure, performance of all steps in thegowning/gloving procedure, performance of a proper order of the steps inthe gowning/gloving procedure, or a combination thereof.

Accordingly, the present invention provides a training system thatincludes a non-transitory memory comprising processor-executableinstructions; and a processor coupled to the non-transitory memory andconfigured to execute the processor-executable instructions. Theprocessor-executable instructions generally comprise instructions to:generate an augmented or virtual reality simulation of an operating roomenvironment; detect, using at least one sensor, an occurrence of anaction within the operating room environment; compare the action to aperformance metric; and generate, based at least in part on thecomparing, a performance evaluation, wherein the performance metricidentifies actions taken during one or more of a scrubbing-in procedure,a gowning/gloving procedure, before a surgical procedure, during thesurgical procedure, and after the surgical procedure.

The actions may be movements of the user, such as grasping a virtualscrub brush to clean the hands, or may be selections, such as using aperipheral control device (e.g., joystick, etc.) or by aiming the usersgaze at a specific selectable spot within the display, such as pointingat the scrub brush or at a selection on a drop down menu. Moreover, thevirtual display may represent the user in the third person, or in thefirst person, or may use a combination thereof through various portionsof the training session.

As shown in FIG. 4, a system 100 of the present invention may include aprocessor 110, a memory 160, and a display 115. In various embodiments,the system 100 may additionally include one or more secondary processors110 a, one or more secondary displays 115 a, a peripheral control 120, aglobal positioning system (GPS) 130, an orientation sensor 140, amicrophone 150, and/or a speaker 155. As shown in FIG. 4, each of theGPS 130, orientation sensor 140, and microphone 150 may be a part of thedisplay 115, such as an AR/VR device, wherein the display 115 may be inelectronic communication with either or both of the processors (110, 110a). Moreover, when included, the peripheral control 120 may be inelectronic communication with the display 115 and/or the processor(s)(110, 110 a).

The peripheral control 120 may refer to a remote control, such as ahand-held unit that may provide allow manual selection (e.g., viabuttons or IR) of options. In some embodiments, the peripheral control120 includes a joystick. The orientation sensor 140 determines thegyroscopic orientation of the user and/or display unit 115, and mayenable the system to determine the angle the user is looking. The GPS130 may be included to further aid in detecting movement of the userand/or display unit 115. The orientation sensor 140 and/or GPS 130 maybe included on a plurality of suitable display devices (AR/VR devices).

The microphone 150 may enable the user to provide auditory cues whenapplicable to tasks performed in/on the virtual OR environment. Theauditory cues received by the microphone 150 may be processed by thesystem and may be a source of simulation data. The speaker 155 mayenable the user to receive auditory cues when applicable to tasksperformed in/on the virtual OR environment.

Additional elements of the system of the present invention may include amotion tracker 172 and eye tracker 174, which may be provided to improvethe immersiveness of the virtual OR environment and provide contextualdata for actions performed by the user within the virtual ORenvironment. Moreover, one or more additional sensors (200, 200 a) maybe included as part of the VR/AR device, or separate from the VR/ARdevice. These additional sensors may be in electronic communication withthe processor 110, and may provide additional information that mayassist in tracking movement of the user or real objects in the augmentedor virtual OR environment, or may assist in defining the OR environment(e.g., camera that may view the actual OR environment in an augmentedreality simulation).

The memory 160 may be associated with the processor 110 and may storedata collected by sensors (200, 200 a) associated with andcommunicatively coupled to the display device or AR/VR device. Thememory 160 may further store the processor-executable instructions usedto execute the methods of the present invention. The memory 160 mayadditionally contain a performance metric of best practices for the userin each of the simulated procedures (e.g., scrubbing-in,gowning/gloving). The actions of the user in the virtual OR environmentmay be compared to and judged against this metric.

While not shown in FIG. 4, when included, the peripheral display 115 amay also include a peripheral control and/or peripheral sensors.

The action within the operating room environment may comprise a physicalmovement of a user of the system, a movement of a real object in theoperating room environment, a movement of a virtual object in theoperating room environment, or any combination thereof.

The display 115 may be configured to provide a visual representation ofthe operating room environment. According to certain aspects, thedisplay 115 may be configured to provide a visual feedback of theperformance evaluation.

According to certain aspects, the display 115 may be an AR/VR device.The AR/VR device may include one or more screens, such as a singlescreen or two screens (e.g., one per eye of a user). The screens mayallow light to pass through the screens such that aspects of the realenvironment are visible while displaying a virtual object. The virtualobject may be made visible to the user by projecting light. The virtualobject may appear to have a degree of transparency or may be opaque(i.e., blocking aspects of the real environment).

A user of the system may interact with the virtual object, such as bymoving the virtual object from a first position to a second position.For example, the user may move an object with his or her hand. This maybe done in the AR/VR system virtually by determining that the hand hasmoved into a position coincident or adjacent to the object (e.g., usingone or more sensors 200, which may be mounted on an AR/VR device, andwhich may be static or may be controlled to move), and causing theobject to move in response. Virtual aspects may include virtualrepresentations of real world objects, drop-down menus, text, and/or mayinclude visual effects, such as lighting effects, etc.

The system and AR/VR device may provide an entirely virtualrepresentation of an OR environment. As such, no portion of thereal-world environment may be included in the systems and methods of thepresent invention.

The OR environment provided by the system may be viewable to one or moreviewers, for example, each viewer using their own AR/VR device, and mayinclude differences among views available for the one or more viewerswhile retaining some aspects as universal among the views. For example,a heads-up display may change between two views while virtual objectsmay be fixed to a real object or area in both views. Aspects such as acolor of an object, lighting, or other changes may be made among theviews without changing a fixed position of at least one virtual object.Such may be used to provide a more realistic representation of a virtualenvironment, and/or may be used to provide a different set of actionresponses. For example, one user may be designated as a trainer andanother user may be designated as an observer (e.g., trainer).

The system may include rules to govern the behavior of virtual objects,such as subjecting a virtual object to gravity or friction, or mayinclude other predefined rules that defy real world physical constraints(e.g., floating objects, perpetual motion, etc.).

According to certain aspects of the system, the OR environment mayinclude a space outside of the operating room, such as a “sink room, andan operating room. In general, the scrubbing-in procedure occurs in thespace outside of the operating room, and may include the 10 stepsoutlined above. Moreover, the gowning/gloving procedure generally occursin the operating room (OR), and may include the 12 steps outlined above.

The system of the present invention may provide a visual, audible, orhaptic feedback on the performance evaluation (e.g., alerts). Moreover,when the feedback is visual and/or audible, it may also provide acorrective instruction after an incorrectly executed action has beendetected by the system (e.g., text instruction presented on the display,audible instruction, etc.).

The system of the present invention may be configured to provide areal-time performance score and/or a final performance score based on acumulative performance evaluation.

According to certain aspects, the system may provide a virtual realityrepresentation of the OR environment. As such, the processor-executableinstructions comprise instructions to generate a virtual realitysimulation of the operating room environment, and the performance metricmay identify movement of the virtual objects in the operating roomenvironment. Movement of the virtual objects in the operating roomenvironment may be indicated by selections from the user, such asprovided by user gestures, manual selection, and/or audible input of theuser.

Detecting actions on objects within the augmented or virtual realityenvironment may include use of at least one sensor. The sensor(s) maydetect actions of both real and virtual objects within the environment.These actions may be compared to a performance metric. For example, theperformance metric may include data related to any of the steps detailedabove (e.g. a length of time to properly wash hands, categories ofsterile and non-sterile objects, proper steps in a gowning procedure,etc.). Thus, for example, if a user touches a non-sterile surface afterwashing their hands, or donning gloves, that action may be recorded andindicated in a performance evaluation.

Moreover, the augmented/virtual reality environment may include virtualversions of real objects, where the virtual object occupies the samespace as the real object. Thus, actions on virtual versions of realobjects in the augmented or virtual reality environment may be compared,and actions and interactions with these virtual versions may beevaluated to accurately change the appearance or other properties of thevirtual objects as defined by pre-programmed properties. As such, visualsignals related to a user's actions may be shown on the virtual versionsof the real objects (e.g., change color of non-sterile/sterile objects,etc.).

This type of feedback may also be included in purely virtualenvironments. Thus, the appearance or properties of purely virtualobjects, with no physical analog in the real environment, may be changedbased upon actions and interactions with other purely virtual objects orreal objects, as defined by their pre-programmed properties. This couldinclude purely visual changes, movements, or other properties such asaudio vocalizations.

Detection of actions and interactions may involve the use of wearable orfreestanding sensors, such as cameras, infrared (IR) beacons, wirelessbeacons, and inertial measurement units. Some sensors may be attached toaugmented reality devices worn by participants or users, or they may beattached to real objects and communicate with the system to provideadditional information about the state of the real and virtual spacerelative to those objects, or they may be freestanding.

Recording the events transpiring in an augmented reality environmentrefers to a method of detecting and recording user actions such as bodymovement, and speech in addition to the passage of time, the visual oraudio experience of participants, and the occurrence of pre-programmedevents, and using that record to evaluate user performance based up onpre-determined metrics.

The sensor(s) may be communicatively coupled to a computer. Thesensor(s) may be configured to provide data (e.g., image data, senseddata, six degrees of freedom data, etc.) to the computer. Furthermore,the sensor(s) may be configured to receive data (e.g., configurationdata, setup data, commands, register settings, etc.) from the computer.The computer may include one or more processors, memory devices, andstorage devices. The processor(s) may be used to execute software, suchas the training software disclosed above, image processing software,sensor(s) software, and so forth. Moreover, the processor(s) may includeone or more microprocessors, such as one or more “general-purpose”microprocessors, one or more special-purpose microprocessors and/orapplication specific integrated circuits (ASICS), or some combinationthereof. For example, the processor(s) may include one or more reducedinstruction set (RISC) processors.

The storage device(s) (e.g., nonvolatile storage) may include ROM, flashmemory, a hard drive, or any other suitable optical, magnetic, orsolid-state storage medium, or a combination thereof. The storagedevice(s) may store data (e.g., data corresponding to a trainingoperation, video and/or parameter data corresponding to a trainingoperation, etc.), instructions (e.g., software or firmware for thetraining system, the sensor(s), etc.), and any other suitable data.

The memory device(s) may include a volatile memory, such as randomaccess memory (RAM), and/or a nonvolatile memory, such as read-onlymemory (ROM). The memory device(s) may store a variety of informationand may be used for various purposes. For example, the memory device(s)may store processor-executable instructions (e.g., firmware or software)for the processor(s) to execute, such as instructions for a trainingsimulation and/or for the sensor(s). In addition, a variety of controlregimes for various training processes, along with associated settingsand parameters may be stored in the storage device(s) and/or memorydevice(s), along with computer code configured to provide a specificoutput (e.g., feedback related to performance metric or evaluation,etc.) during operation.

The programs and systems of the present invention may be configured towork across many platforms (e.g., Android, iOS, Microsoft Windows, UNIX,etc.). Moreover, the presently disclosed software applications may useprogramming of one or both of the graphics processing unit (e.g., CUDA,DirectX or OpenCL) and the central processing unit to enhance thecomputational performance.

Active learning has been shown to be more effective than passivelearning. As such, the methods and systems of the present inventionprovide improved means to learn various procedures, such as thescrubbing-in procedure, gowning/gloving procedure, and/or OR etiquette,by creating an active learning environment. Moreover, the methods andsystems of the present invention provide an enjoyable and helpful meansto learn these procedures, and provides all parties in the OR involvedwith the application/device—OR novices, OR experts, the patient,etc.—with a competitive advantage over OR rooms that have personnel thathave not been trained by the methods and systems of the presentinvention. The novices will be better prepared for the OR, which willlead to a more sterile, comfortable OR with more fluid, efficientprocedures. The improved sterility will decrease the risk ofcontamination of the patient, which improves patient outcomes andcost/time effectiveness (i.e. less adverse event reports/costs). Theincreased efficiency will make the surgery more cost effective due tothe possible reduced time of the procedure. The improved comfort (due tothe increased OR knowledge bases of the OR novices) will provide a lesstense environment that ultimately reduces the overall stress of the ORfor all parties involved.

The methods and systems disclosed herein may also find use beyondtraining, such as medical education, and medical certification.Additionally, the methods and systems of the present invention may beconfigured as games, wherein positive feedback may gain rewards andnegative feedback may reduce reward (e.g., score).

As such, while specific embodiments of the invention have been describedin detail, it should be appreciated by those skilled in the art thatvarious modifications and alternations and applications could bedeveloped in light of the overall teachings of the disclosure.Accordingly, the particular arrangements, systems, apparatuses, andmethods disclosed are meant to be illustrative only and not limiting asto the scope of the invention.

What is claimed is:
 1. A training system comprising: a non-transitorymemory comprising processor-executable instructions; and a processorcoupled to the non-transitory memory and configured to execute theprocessor-executable instructions, wherein the processor-executableinstructions comprise instructions to: generate an augmented or virtualreality simulation of an operating room environment; detect, using atleast one sensor, an occurrence of an action within the operating roomenvironment; compare the action to a performance metric; and generate,based at least in part on the comparing, a performance evaluation,wherein the performance metric identifies actions taken during one ormore of a scrubbing-in procedure, a gowning/gloving procedure, before asurgical procedure, during the surgical procedure, and after thesurgical procedure.
 2. The training system of claim 1, wherein theaction within the operating room environment comprises a physicalmovement of a user of the system, a movement of a real object in theoperating room environment, a movement of a virtual object in theoperating room environment, or a combination thereof.
 3. The trainingsystem of claim 1, further comprising: a display in communication withthe processor, wherein the display is configured to provide a visualrepresentation of the operating room environment.
 4. The training systemof claim 3, wherein the display is further configured to provide avisual feedback of the performance evaluation.
 5. The training system ofclaim 1, wherein the performance evaluation defines an aseptic qualityof the scrubbing-in procedure, a speed of the scrubbing-in procedure,performance of all steps in the scrubbing-in procedure, performance of aproper order of the steps in the scrubbing-in procedure, or acombination thereof.
 6. The training system of claim 1, wherein theperformance evaluation defines an aseptic quality of the gowning/glovingprocedure, a speed of the gowning/gloving procedure, performance of allsteps in the gowning/gloving procedure, performance of a proper order ofthe steps in the gowning/gloving procedure, or a combination thereof. 7.The training system of claim 1, wherein the operating room environmentincludes a space outside of the operating room and an operating room. 8.The training system of claim 7, wherein the scrubbing-in procedurecomprises the following steps performed in the space outside of theoperating room: removing jewelry from hands and arms; putting on a facemask and eye protection; taking and opening a pre-packaged scrub/nailkit; moistening hands and arms with water from a faucet without touchingthe faucet; using a bristled side of a brush from the pre-packagedscrub/nail kit to scrub nails; using a bristled end of the brush toscrub all surfaces of fingers and hands; using a sponge to scrub anentire length of one forearm, from wrist to elbow; using the sponge toscrub an entire length of contralateral forearm, from wrist to elbow;rinsing off both forearms allowing water to fall from wrists to elbows;and using back, butt, or hip to enter the operating room while keepingarms positioned up and away from torso.
 9. The training system of claim1, wherein the gowning/gloving procedure comprises: entering theoperating room environment with cleaned wet hands and arms in anelevated position, and approaching a scrub technician or nurse, takingcare to avoid touching anything; holding out first hand to accept a drytowel from the scrub technician or nurse; drying second hand and armusing the first hand, from hand to elbow; passing the towel to thesecond hand to dry the first hand and arm, from hand to elbow; placingboth hands and arms into sleeves of a gown, allowing fingertips to comethrough ends of sleeves, wherein the gown is held open by the scrubtechnician or nurse; allowing a nonsterile nurse or circulator to tie aback of the gown using a first set of ties; placing right hand intoright glove, wherein the right glove is held open by the scrubtechnician or nurse; placing left hand into left glove, wherein the leftglove is held open by the scrub technician or nurse; handing a secondtie to the scrub technician or nurse, or to the nonsterile nurse orcirculator; rotating in gown with the scrub technician or nurse, or tothe nonsterile nurse or circulator still holding the second tie;re-grasping the second tie from the scrub technician or nurse, or to thenonsterile nurse or circulator; and tying the second tie at front of thegown.
 10. The training system of claim 1, wherein the system providesvisual, audible, or haptic feedback on the performance evaluation. 11.The training system of claim 10, wherein the visual feedback, audiblefeedback, or both provide a corrective instruction after an incorrectlyexecuted action.
 12. The training system of claim 1, wherein the systemprovides a final performance score based on a cumulative performanceevaluation.
 13. The training system of claim 1, wherein theprocessor-executable instructions comprise instructions to generate avirtual reality simulation of the operating room environment, and theperformance metric identifies movement of the virtual objects in theoperating room environment.
 14. The training system of claim 13, whereinmovement of the virtual objects in the operating room environment areindicated by selections from a user of the system, wherein theselections are provided by user gestures, manual or audible input of theuser, or a combination thereof.
 15. A training method for a scrubbing-inprocedure, the method comprising: identifying, using a processor, ascrubbing-in procedure including a plurality of steps; presenting, usingan augmented reality display of an augmented reality device, a virtualrepresentation of a sink room of an operating room suite; querying auser for a first step in the scrubbing-in procedure; determining whetherthe first step in the scrubbing-in procedure has been completed; and inresponse to determining that the first step has been completed,presenting to the user a virtual representation of a subsequent step inthe scrubbing-in procedure.
 16. The method of claim 15, furthercomprising: before determining the first step in the scrubbing-inprocedure has been completed: determining that the first step is notbeing performed, or that the first step is being performed improperly;and displaying an alert using the augmented reality display indicatingthat an error.
 17. The method of claim 15, wherein the plurality ofsteps in the scrubbing-in procedure are ordered, and include: removingjewelry from hands and arms; putting on a face mask and eye protection;taking and opening a pre-packaged scrub/nail kit; moistening hands andarms with water from a faucet without touching the faucet; using abristled side of a brush from the pre-packaged scrub/nail kit to scrubnails; using a bristled end of the brush to scrub all surfaces offingers and hands; using a sponge to scrub an entire length of oneforearm, from wrist to elbow; using the sponge to scrub an entire lengthof contralateral forearm, from wrist to elbow; rinsing off both forearmsallowing water to fall from wrists to elbows; and using back, butt, orhip to enter the operating room while keeping arms positioned up andaway from torso.
 18. The method of claim 15, further comprising, afterthe plurality of steps in the scrubbing-in procedure are complete:querying a user for a first step in a gowning/gloving procedure, whereinthe gowning/gloving procedure is presented on the augmented realitydisplay of the augmented reality device as a virtual representation ofan operating room; determining whether the first step in thegowning/gloving procedure has been completed; and in response todetermining that the first step has been completed, presenting to theuser a virtual representation of a subsequent step in thegowning/gloving procedure.
 19. The method of claim 18, furthercomprising: before determining the first step in the gowning/glovingprocedure has been completed: determining that the first step is notbeing performed, or that the first step is being performed improperly;and displaying an alert using the augmented reality display indicatingthat an error.
 20. The method of claim 18, wherein the plurality ofsteps in the gowning/gloving are ordered, and include: entering theoperating room environment with cleaned wet hands and arms in anelevated position, and approaching a scrub technician or nurse, takingcare to avoid touching anything; holding out first hand to accept a drytowel from the scrub technician or nurse; drying second hand and armusing the first hand, from hand to elbow; passing the towel to thesecond hand to dry the first hand and arm, from hand to elbow; placingboth hands and arms into sleeves of a gown, allowing fingertips to comethrough ends of sleeves, wherein the gown is held open by the scrubtechnician or nurse; allowing a nonsterile nurse or circulator to tie aback of the gown using a first set of ties; placing right hand intoright glove, wherein the right glove is held open by the scrubtechnician or nurse; placing left hand into left glove, wherein the leftglove is held open by the scrub technician or nurse; handing a secondtie to the scrub technician or nurse, or to the nonsterile nurse orcirculator; rotating in gown with the scrub technician or nurse, or tothe nonsterile nurse or circulator still holding the second tie;re-grasping the second tie from the scrub technician or nurse, or to thenonsterile nurse or circulator; and tying the second tie at front of thegown.